The Multivalued and Continuous Perceptrons

Rosenblatt's perceptron is extended to (1) a multivalued perceptron and (2) to a continuous-valued perceptron. It shown that any function that can be represented by the multivalued perceptron can be learned in a nite number of steps, and any function that can represented by the continuous perceptron can be learned with arbitrary accuracy in a nite number of steps. The whole apparatus is deened in the complex domain. With these perceptrons learnability is extended to more complicated functions than the usual linearly separable ones. The complex domain promises to be a fertile ground for neural networks research. One of the most remarkable and important theorems in the neural networks literature is Rosenblatt's Perceptron Theorem Ros62]. Simply stated, it says that if there is a weight vector W such that W T X > 0 whenever pattern vector X belongs to class ! 1 and W T X < 0 whenever X belongs to class ! 0 , i.e. the two classes are linearly separable, then a given simple learning algorithm will nd a weight vector W that will satisfy the above two inequalities. The superscript T indicates matrix transpose. In other words, if the pattern vectors, which belong to two classes ! 0 and ! 1 , are linearly separable, then the algorithm will nd a hyperplane that separates the two classes. The activation function used is the hard thresholding function, which gives two outputs: ?1 or 1. In this work Rosenblatt's Perceptron Theorem is generalized to (1) the Multivalued Perceptron, where the output can take more than two discrete values, and (2) the Continuous Perceptron, where the output takes continuous values. In both cases all quantities involved are complex-valued. The output of the activation functions falls on the unit circle in the complex plane. Rosenblatt's perceptron is a special case of both perceptrons. The activation functions were previously used in Noe88a] and Noe88b] in a diierent context, and are diierent from the one used for the complex backpropagation algorithm in Geo92b, GK92].